Video camera with rate control video compression

ABSTRACT

Embodiments provide a video camera that can be configured to compress video data in a manner that achieves a targeted output size in a computationally efficient manner. The video compression systems and methods can be used with DCT-based compression standards to include a rate control aspect. The rate controlled video compression methods can be configured to compress video data in real time and/or using a single pass. During compression of video data, the video compression systems and methods can modify compression parameters to achieve a targeted file size while maintaining relatively high visual quality of the compressed images.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/092,470, entitled “Video Camera with Rate Control Video Compression,”filed Apr. 6, 2016, which claims the benefit of priority to U.S. Prov.Pat. App'n No. 62/145,969, entitled “Video Camera with Rate ControlVideo Compression,” filed Apr. 10, 2015; the entire contents of whichare incorporated by reference herein for all purposes.

BACKGROUND Field

The present disclosure is directed to digital cameras, such as those forcapturing moving pictures, and more particularly, to digital camerasthat compress video data.

Description of Related Art

With the availability of digital video cameras, producers of majormotion pictures, television broadcast media, commercials and other videocreators can capture and edit digital video. Capturing high qualityvideo (e.g., high frame rate and/or high resolution) can require arelatively large amount of digital storage. Compressing high qualityvideo can be computationally expensive and/or time consuming.

SUMMARY

Although some currently available digital video cameras include highresolution image sensors, and thus are capable of recording, storing,and/or providing high resolution video, the image compression techniquesused on board such cameras can be computationally expensive. Somedigital cameras do not compress video on board the camera, and theresulting video files can become relatively large compared to compressedvideo. Such digital video cameras can be limited in the amount ofuncompressed video that can be stored onboard and/or can be expensivedue to the large amounts of digital storage built into the camera. Somedigital cameras compress video data using compression algorithms thatare not compatible with typical or widely-used video editing software.For cameras that do not store video in a compressed format compatiblewith video editing software, when the captured video is to be edited,the stored video data generally needs to be compressed using a standardcompression algorithm. The additional step of applying these compressionalgorithms to stored video data can add complexity to work flow, can betime-consuming, and/or can be frustrating to users of the video camera.Some digital cameras compress video on board the camera, but do so in acomputationally expensive or inefficient manner. Such cameras canrequire expensive computing hardware to handle these compression tasks.Even where video is compressed on board, the compressed video may stillbe relatively large, requiring a large amount of digital storage.Accordingly, an aspect of some embodiments disclosed herein includes avideo camera that is configured to perform video compression on boardutilizing techniques that are relatively computationally efficient, thatcan be done in real time, that can generate video files ready for videoediting, and/or that can have a relatively small file size.

Thus, in accordance with some embodiments, a video camera can comprise aportable housing and a lens assembly supported by the housing andconfigured to focus light. A light sensitive device and associatedelectronics can be configured to convert the focused light into rawimage data. The camera can also include a memory device, an imageprocessing device, and a video compression device, the devicescollectively and/or individually configured to process the raw and/orcompressed image data, to compress the processed and/or raw image data,and to store the compressed video data in the memory device. In someimplementations, the video compression device can be configured to applya rate control algorithm operating with a discrete cosine transform(“DCT”) to generate compressed video data in real time.

In accordance with certain embodiments, a method of recording video witha video camera can include directing light onto an image sensor. Themethod can also include converting the light received by the imagesensor into raw digital image data, processing the raw digital imagedata, and compressing the processed image data using a rate controlalgorithm operating with a DCT, and recording or storing the compressedimage data onto a storage device.

In accordance with some embodiments, a video camera can comprise a lensassembly supported by the housing and configured to focus light and alight sensitive device configured to convert the focused light into asignal of raw image data representing the focused light. The camera canalso include a memory device and a video compression device forcompressing and recording the compressed image data at a frame rate ofat least about 30 frames per second. In a further implementation, thecamera can be configured to compress and record compressed image data ata frame rate of at least about 50 frames per second or at least about 60frames per second. In some implementations, the camera can be configuredto compress and record compressed image data at a frame rate of at leastabout 30 frames per second where each frame of raw image data includesat least 1280 horizontal pixels. In some implementations, the camera canbe configured to compress and record compressed image data at a framerate of at least about 23 frames per second, at least about 30 framesper second, at least about 50 frames per second, or at least about 60frames per second where individual frames of raw image data include atwo-dimensional array of pixels with at least 1280 pixels in onedimension, at least 1920 pixels in one dimension, at least 3840 pixelsin one dimension, or at least 4000 pixels in one dimension. In someimplementations, the camera can be configured to compress and recordcompressed image data at a frame rate of at least about 23 frames persecond, at least about 30 frames per second, at least about 50 framesper second, or at least about 60 frames per second where individualframes of raw image data include a two-dimensional array of pixels withat least 1280×720 pixels, at least 1920×1080 pixels, at least 3840×2160pixels, or at least 4000×3000 pixels. In some implementations, the videocompression device can be configured to compress and store in the memorydevice the compressed image data using a single-pass compressionalgorithm that does not utilize frame memory during compression. Incertain implementations, the video compression device can compress theraw image data at a rate of at least about 4 frames per second, at leastabout 23 frames per second, at least about 24 frames per second, atleast about 30 frames per second, at least about 50 frames per second,or at least about 60 frames per second.

In accordance with some embodiments, a video camera can comprise aportable housing having at least one handle configured to allow a userto manipulate the orientation with respect to at least one degree ofmovement of the housing during a video recording operation of thecamera. A lens assembly can comprise at least one lens supported by thehousing and configured to focus light at a plane disposed inside thehousing. A light sensitive device can be configured to convert thefocused light into raw image data with a horizontal resolution of atleast 1280 pixels and at a frame rate of at least about 4 frames persecond. A memory device can also be configured to store compressed videoimage data. An image processing system can be configured to process theraw video image data. A video compression system can be configured tocompress and store in the memory device the compressed image data usinga single-pass compression algorithm that does not utilize frame memoryduring compression, wherein the video compression system can compressimage data at a rate of at least about 4 frames per second.

Another aspect of some embodiments disclosed herein includes therealization that a single-pass compression algorithm can be implementedthat complies with discrete cosine transform (DCT) compressionstandards. Because the single-pass compression algorithm complies withcertain DCT compression standards, the output of the compressionalgorithm can be compatible with video editing software that acceptsvideo compressed using DCT compression standards (e.g., the Apple®ProRes family of video codecs, MPEG-2, H.264, MPEG-4, MJPEG, DV, Daala,Theora, etc.).

Another aspect of some embodiments disclosed herein includes therealization that a single-pass video compression algorithm can beimplemented that modifies compression parameters to achieve a targetframe size and/or to achieve a frame size that is less than or equal toa maximum video frame size for each video frame (e.g., where size hererefers to the amount of digital data used to represent the compressedvideo frame). The compression algorithm can be performed in a singlepass due at least in part to the compression parameters being calculatedusing results of compression of a previous section(s) of a frame and/ora previous video frame(s). This allows the single-pass compressionalgorithm to be used where bandwidth limits dictate video bit rates,which are related to the number of bytes per frame of compressed video,such as in conjunction with broadcasting standards. This also allows thecompression algorithm to operate without utilizing frame memory duringcompression (e.g., previous frames are not put into a frame memory forutilization during compression of the current frame).

In some embodiments, a video compression system can compress video databy processing individual video frames. The video compression system candivide an individual video frame into a plurality of sections and candivide each section into a plurality of slices. In some embodiments,slices can be further divided into macroblocks and macroblocks can bedivided into blocks. In typical applications, a block size can be 8×8pixels and a macroblock size can be 16×16 pixels, or 4 blocks. Differentsizes of blocks and macroblocks can also be used. For each slice withina section, the compression algorithm can apply a DCT to determine DCTcoefficients for the slice. This can be accomplished, for example, byapplying a two-dimensional DCT on each block in the slice. The amount ofinformation in the slice can be used to determine compression parametersthat can be used to adjust a quantization table used to quantize the DCTcoefficients. When the slices in a particular section have beencompressed, the current size of the compressed frame can be compared toa targeted frame size, wherein the comparison takes into account thefraction of the frame that has already been processed, to adjustcompression parameters used in the DCT. When all of the sections of avideo frame have been compressed, the size of the compressed video framecan be compared to a targeted or maximum frame size. Based at least inpart on this comparison, compression parameters can be modified forcompression of the following video frame. Accordingly, the compressionalgorithm can be configured to modify or adjust compression parametersduring compression of a video frame to achieve a targeted frame size.Similarly, the compression algorithm can be configured to modify oradjust compression parameters from frame to frame to achieve a targetedframe size.

In some embodiments, the video compression system can be implemented aspart of a video camera. The video compression system can be implementedusing hardware, software, or a combination of both hardware andsoftware. In some embodiments, the video compression system can beimplemented separate from a video camera. For example, the videocompression system can be part of a video editing suite of programs. Incertain implementations, functionality provided by the video compressionmethods disclosed herein is included in a software development kit.

The video compression systems and methods disclosed herein can berelatively computationally efficient. As used herein, computationallyefficient can refer to an efficient use of processor cycles, processorpower, memory, or the like. Thus, a procedure is relativelycomputationally efficient where it utilizes fewer computationalresources (e.g., processor cycles, processor power, memory, etc.) thananother procedure. This can result in reduced-cost video cameras. Thesingle-pass algorithm can also be configured to compress video in realtime at relatively high frame rates (e.g., at least 23 frames persecond, at least about 24 frames per second, at least about 30 framesper second, at least about 50 frames per second, at least about 60frames per second, etc.). The single-pass algorithm can also beconfigured to compress video in real time at relatively high resolutions(e.g., at least 1280 horizontal pixels, at least about 1920 horizontalpixels, at least about 3840 horizontal pixels, at least about 4000horizontal pixels, at least about 6000 horizontal pixels, etc.).

The video compression systems and methods disclosed herein provide anumber of advantages. For example, the disclosed video compression canbe configured to provide real-time video compression at relatively highframe rates and/or relatively high resolutions. The disclosed videocompression can be configured to compress video using a standard formatfor use with video editing tools. The disclosed video compression can beconfigured to compress video data in real time for storage on board avideo camera. The disclosed video compression can be configured toreduce a file size of compressed video relative to other videocompression schemes that utilize DCTs. The disclosed video compressioncan be configured to provide the advantage of a reduced file size whilealso maintaining or surpassing the visual quality of the compressedvideo frames relative to those other video compression schemes. Thedisclosed video compression can take advantage of similarity betweenconsecutive frames to reduce compressed video frame sizes. The disclosedvideo compression can account for scene changes by making intra-frameadjustments to compression parameters. The disclosed video compressioncan implement multi-tiered compression techniques. For example, when thenumber of bits generated for a portion of a video frame exceeds a targetsize, another tier of rate controls can be utilized so that the finalvideo frame size can be less than a desired, selected, targeted, ormaximum video frame size. The disclosed video compression can beperformed in a single pass, thereby reducing computational costs. Thedisclosed video compression can be configured to adjust compressionparameters based at least in part on compression results within a frameof image data (e.g., a rate control compression algorithm). Thedisclosed video compression can be configured to adjust compressionparameters based at least in part on compression results from one ormore previous frames of image data (e.g., a rate control compressionalgorithm).

BRIEF DESCRIPTION OF THE DRAWINGS

The following figures are used to illustrate example embodiments of thesystems and methods disclosed herein and are not intended to limit thescope of the disclosure. Throughout the drawings, like numbers can beused to refer to like elements.

FIG. 1 illustrates a schematic diagram of a video camera configured toimplement video compression with rate control.

FIG. 2 illustrates an example embodiment of a video camera configured toimplement video compression with rate control.

FIG. 3 illustrates a block diagram of a process of dividing up videodata into sections and slices for video compression with rate control.

FIG. 4 illustrates a flow chart of an example method of videocompression with rate control.

FIG. 5 illustrates a flow chart of an example method of compressing aslice of a video frame.

FIG. 6 illustrates a flow chart of an example method of adjusting videocompression parameters after compressing a section of a video frame.

FIG. 7 illustrates a flow chart of an example method of adjusting videocompression parameters after compressing a video frame.

FIG. 8 illustrates a flow chart of an example method of adjusting videocompression parameters based on an accumulated size of a compressedvideo frame while compressing the video frame.

FIG. 9 illustrates a flow chart of an example method of adjusting videocompression parameters based on an accumulated size of a compressedvideo frame relative to an accumulated size of a previous frame orframes after the same section within the frames.

DETAILED DESCRIPTION

Disclosed herein are video compression systems and methods that canadjust compression parameters during compression to achieve targetedfile sizes, video frame sizes, video bit rates, and the like. The videocompression systems and methods are described herein as being compatiblewith DCT-based compression standards. Thus, the output of the describedvideo compression systems and methods can be compatible with tools,software, and/or programs configured to operate on files compressedusing DCT-based compression techniques. For example, decoders configuredto decode video compressed using DCT-based compression standards can becapable of decoding video compressed with the video compression methodsand systems disclosed herein. Examples of DCT-based video compressionstandards that may be compatible with the video compression systems andmethods described herein include, without limitation, JPEG, MJPEG,Theora, MPEG-1, MPEG-2, H.261, H.263, H.264/AVC, H.265/HEVC, etc.

Example Video Cameras

FIG. 1 illustrates a schematic diagram of a video camera 10 that caninclude a body or housing 12 configured to support a system 14configured to detect, process, compress, store and/or replay video imagedata. For example, the system 14 can include optics hardware 16, animage sensor 18, an image processing system 20, a compression system 22,and a storage device 24. In some implementations, the video camera 10can also include a monitor system 26, a playback system 28, and adisplay 30.

FIG. 2 illustrates an exemplary embodiment of the video camera 10. Theoptics hardware 16 can be supported by the housing 12 in a manner thatleaves a portion of the optics hardware 16 exposed at an outer surfaceof the housing 12. In some embodiments, the system 14 is supportedwithin the housing 12. For example, the image sensor 18, imageprocessing system 20, and the compression system 22 can be housed withinthe housing 12. The storage device 24 can be mounted in the housing 12.In some embodiments, the storage device 24 can be mounted to an exteriorof the housing 12 and connected to the remaining portions of the system14 through any type of known connector or cable. Additionally, thestorage device 24 can be connected to the housing 12 with a flexiblecable, thus allowing the storage device 24 to be moved somewhatindependently from the housing 12. For example, with such a flexiblecable connection, the storage device 24 can be worn on a belt of a user,allowing the total weight of the housing 12 to be reduced. In certainembodiments, the housing 12 can include one or more storage devices 24inside and mounted to its exterior. The housing 12 can be configured tosupport the monitor system 26 and playback system 28. In certainembodiments, the display 30 can be configured to be mounted to anexterior of the housing 12.

The optics hardware 16 can be in the form of a lens system having atleast one lens configured to focus an incoming image onto the imagesensor 18. In some embodiments, the optics hardware 16 can be in theform of a multi-lens system providing variable zoom, aperture, andfocus. The optics hardware 16 can be in the form of a lens socketsupported by the housing 12 and configured to receive a plurality ofdifferent types of lens systems for example, but without limitation, theoptics hardware 16 can include a socket configured to receive varioussizes of lens systems including a 50-100 millimeter (F2.8) zoom lens, an18-50 millimeter (F2.8) zoom lens, a 300 millimeter (F2.8) lens, 15millimeter (F2.8) lens, 25 millimeter (F1.9) lens, 35 millimeter (F1.9)lens, 50 millimeter (F1.9) lens, 85 millimeter (F1.9) lens, and/or anyother lens. As noted above, the optics hardware 16 can be configuredsuch that images can be focused upon a light-sensitive surface of theimage sensor 18 despite which lens is attached thereto.

The image sensor 18 can be any type of video sensing device, including,for example, but without limitation, CCD, CMOS, vertically-stacked CMOSdevices such as the Foveon® sensor, or a multi-sensor array using aprism to divide light between the sensors. In some embodiments, theimage sensor 18 can include a CMOS device having about 12 millionphotocells. However, other size sensors can also be used. In someconfigurations, video camera 10 can be configured to output video at “2k” (e.g., 2048×1152 pixels), “4 k” (e.g., 4,096×2,540 pixels), “4.5 k,”“5 k,” “6 k,” and/or “8 k” or greater resolutions. As used herein, inthe terms expressed in the format of “xk” (such as “2 k” and “4 k” notedabove), the “x” quantity refers to the approximate horizontalresolution. As such, “4 k” resolution corresponds to about 4000 or morehorizontal pixels and “2 k” corresponds to about 2000 or more pixels.Using currently commercially available hardware, the image sensor 18 canbe as small as about 0.5 inches (8 mm), but it can be about 1.0 inches,or larger. Additionally, the image sensor 18 can be configured toprovide variable resolution by selectively outputting only apredetermined portion of the image sensor 18. For example, the imagesensor 18 and/or the image processing system 20 can be configured toallow a user to identify, configure, select, or define the resolution ofthe video data output.

The video camera 10 can also be configured to down-sample andsubsequently process the output of the image sensor 18 to yield videooutput at “2 k,” 1080p, 720p, or any other resolution. For example, theimage data from the sensor 18 can be “windowed,” thereby reducing thesize of the output image and allowing for higher readout speeds.However, other size sensors can also be used. Additionally, the videocamera 10 can be configured to up-sample the output of the image sensor18 to yield video output at higher resolutions.

With reference to FIG. 1, the image processing system 20 of the videocamera 10 can be configured to process raw video data provided by theimage sensor 18. For example, the image sensor 18 can be configured toprovide color image data de-mosaiced according to a pattern andcomprising a plurality of picture element values for each of a pluralityof spatially interleaved color channels. The image processing system 20can be configured to process the picture element values to combine,subtract, multiply, divide, or otherwise modify the picture elements togenerate a digital representation of the image data. For example, thedigital representation can be expressed using RGB values, gammacorrected RGB values, Y′CbCr values (e.g., where Y′ is referred to as“luma” and CbCr is referred to as “chroma” components), or the like. Theimage processing system 20 can include other systems or modules and/orcan be configured to perform other processes, such as, for example, butwithout limitation, gamma correction processes, noise filteringprocesses, etc. Examples of functionality provided by the imageprocessing system 20 are described in U.S. patent application Ser. No.14/180,168 filed Feb. 13, 2014 and entitled “Video Camera” (published asU.S. Pub. No. 2014/0226036), which is incorporated herein by referencein its entirety for all purposes.

The video camera 10 can include a compression system 22. The compressionsystem 22 can be in the form of a separate chip or chips (e.g., FPGA,ASIC, etc.), it can be implemented with software and another processor,and/or it can be implemented with a combination of processors, software,and/or dedicated chips. For example, the compression system 22 caninclude a compression chip that performs a compression technique inaccordance with DCT-based codecs. The compression chip can be configuredto include the rate control aspects of the compression techniquesdescribed herein.

The compression system 22 can be configured to compress the image datafrom the image processing system 20 using DCT-based codecs with ratecontrol, aspects of which are described herein in greater detail withreference to FIGS. 3-9. In some embodiments, the compression system 22performs a compression technique that modifies or updates compressionparameters during compression of video data. The modified or updatedcompression parameters can be configured to achieve targeted or desiredfile sizes, video quality, video bit rates, or any combination of these.In some embodiments, the compression system 22 can be configured toallow a user or other system to adjust compression parameters to modifythe quality and/or size of the compressed video output by thecompression system 22. For example, the video camera 10 can include auser interface that allows a user to input commands that cause thecompression system 22 to change compression parameters.

The compression system 22 can be configured to compress the image datafrom the image processing system 20 in real time. The compression system22 can perform compression using a single-pass to compress video frames.This can be used to eliminate the use of an intermediate frame memoryused in some compression systems to perform multiple compression passesand/or to compress a current video frame based on the content from oneor more previous video frames stored in an intermediate frame memory.This can reduce the cost and/or complexity of a video camera withon-board video compression. The compression system 22 can be configuredto compress image data from the image processing system 20 in real timewhen the frame rate of the image data is at least 23 frames per second(“fps”), at least about 24 fps (e.g., 23.976 fps), at least about 25fps, at least about 30 fps (e.g., 29.97 fps), at least about 48 fps, atleast about 50 fps, at least about 60 fps (e.g., 59.94 fps), at leastabout 120 fps, at least about 240 fps, and/or less than or equal toabout 240 fps. The compressed video can thus be sent to a storage device24 and/or the monitor system 26.

The storage device 24 of the video camera can be in the form of any typeof digital storage, such as, for example, but without limitation, harddisks, flash memory, or any other type of memory device. In someembodiments, the size of the storage device 24 can be sufficiently largeto store image data from the compression system 22 corresponding to atleast about 30 minutes of video at 12 megapixel resolution, 12-bit colorresolution, and at 60 fps. However, the storage device 24 can have anysize.

In some embodiments, the storage device 24 can be mounted on an exteriorof the housing 12. Further, in some embodiments, the storage device 24can be connected to the other components of the system 14 throughstandard communication ports, including, for example, but withoutlimitation, IEEE 1394, USB 3.0, IDE, SATA, etc. Further, in someembodiments, the storage device 24 can comprise a plurality of harddrives operating under a RAID protocol. However, any type of storagedevice can be used.

The system 14 includes, in some implementations, a monitor system 26 anda display device 30 configured to allow a user to view video imagescaptured by the image sensor 18 during operation. In some embodiments,the image processing system 20 can include a subsampling systemconfigured to output reduced resolution image data to the monitor system26. For example, such a subsampling system can be configured to outputvideo image data to support “2 k,” 1080p, 720p, or any other resolution.In some embodiments, filters used for de-mosaicing can be adapted toalso perform down-sampling filtering, such that down-sampling andfiltering can be performed at the same time. The monitor system 26 canbe configured to perform any type of de-mosaicing process to the datafrom the image processing system 20. Thereafter, the monitor system 26can output a de-mosaiced image data to the display 30.

The display 30 can be any type of monitoring device. For example, butwithout limitation, the display 30 can be a four-inch LCD panelsupported by the housing 12. For example, in some embodiments, thedisplay 30 can be connected to an infinitely adjustable mount configuredto allow the display 30 to be adjusted to any position relative to thehousing 12 so that a user can view the display 30 at any angle relativeto the housing 12. In some embodiments, the display 30 can be connectedto the monitor system 26 through any suitable video cables such as, forexample but without limitation, HDMI cables, HD-SDI cables, RGB cables,or YCC format video cables.

The video camera 10 can include a playback system 28 that can beconfigured to receive data from the storage device 24 and/or from thecompression system 22, to decompress and to de-mosaic the image data,and to output the image data to the display 30. In some embodiments, themonitor system 26 and the playback system 28 can be connected to thedisplay 30 through an intermediary display controller (not shown). Assuch, the display 30 can be connected with a single connector to thedisplay controller. The display controller can be configured to transferdata from either the monitor system 26 or the playback system 28 to thedisplay 30.

Example Video Compression Methods

FIG. 3 illustrates a block diagram of a process of dividing up videodata into video frames (or sub-frames), sections, and slices forapplication of a rate control video compression algorithm. Dividing thevideo data into video frames, sections, and slices can allow for thevideo compression systems and methods described herein to modifycompression parameters during compression of a video frame to achieve afile size or a frame size that is within a targeted range and/or that isless than a maximum file size or frame size. As used herein, the term“rate control” as used in conjunction with compression algorithmsgenerally refers to a compression algorithm that dynamically modifiesparameters of the algorithm based at least in part on resultingcompression sizes. This dynamic modification can occur based at least inpart on compression sizes within a frame (or sub-frame) of image dataand/or on compression sizes from frame to frame.

Advantageously, the disclosed video compression systems and methods canbe configured to achieve frame sizes within targeted or desired sizeranges in real time. As used herein, compressing video data in real timecan mean that a single frame of video is compressed prior to the nextframe of video being presented for compression. For example, if videodata has a frame rate of about 30 fps, then the video compression systemcan be configured to compress a frame of video in less than about 1/30thof a second.

Similarly, dividing video frames into sections and slices can allow forthe disclosed video compression systems and methods to compress videodata in a single pass. As used herein, compressing video data in asingle pass (e.g., a single-pass compression algorithm) can mean thatthe video compression system compresses a slice, section, and/or videoframe a single time and does not perform multiple compressions ormultiple passes at compressing the single video frame, section, orslice. For example, after quantizing DCT coefficients of a particularslice of a video frame, the video compression system does not generateor calculate quantized DCT coefficients for that same slice of the videoframe again. The disclosed single-pass video compression algorithms canthus be implemented in a system without using frame memory (e.g., anintermediate memory for frames) to store a previous video frame(s).

Video data 300 can be provided as a series of video frames 302, eachvideo frame 302 being an array of picture elements, or pixels. Videoframes 302 can be a sub-frame wherein a sub-frame comprises subset ofthe pixels available in the raw image data, such as the raw image dataacquired with an image sensor. In some implementations, the pixels of avideo frame 302 can form an image. The pixels of a video frame 302 canrepresent intensity (e.g., a gray-level value), colors (e.g., red,green, blue), luma and/or chroma (e.g., Y′CrCb), an alpha channel, orthe like. In some implementations, a particular video frame 302 can havepixel values that represent differences between previous and/orsubsequent video frames. It is to be understood that a video frame 302is not limited to an array of pixel values that represent color and/orintensity at that pixel. The video compression systems and methodsdisclosed herein can be configured to operate on video frames that havebeen processed such that the pixel values represent, for example andwithout limitation, motion estimation between video frames (e.g., motionestimation) and/or estimations based on adjacent or neighboring blocksof pixels (e.g., intra-estimation).

Each video frame 302 can be divided into slices 304. In someimplementations, each slice 304 can have the same number of pixels. Forexample, each slice can be a rectangle of 128×16 pixels, a rectangle of64×16 pixels, or a square of 16×16 pixels. In some embodiments, eachslice 304 can be divided into one or more macroblocks 308. In certainimplementations, each macroblock can be a square of 16×16 pixels. Eachmacroblock 308 can be further divided into blocks 309. In certainimplementations, a block 309 is a square of 8×8 pixels. The macroblock308, in some implementations, can be made up of 4 blocks. Other sizes ofslices can be used, such as slices that have a different number ofhorizontal pixels and/or vertical pixels. Slices 304 can be groupedtogether into sections 306. The sections 306 can be configured to extendacross (e.g., vertically or horizontally) an entire video frame 302. Thesections 306 can be of different sizes (e.g., each section 306 can havea different number of slices 304 and/or a different number of pixels).In some implementations, the video frame 302 is divided into a singlesection 306. In certain implementations, the video frame 302 is dividedinto a single slice 304.

In some embodiments, each slice 304 of the video frame 302 can have itsown quantization table. In certain implementations, the quantizationtable of each slice 304 of the video frame 302 can be generated by usinga function to transform a fixed or standard quantization table. Forexample, the fixed or standard quantization table can be multiplied by avariable value.

FIG. 4 illustrates a flow chart of an example method 400 of videocompression with rate control. In some embodiments, the method 400 canrepresent a control routine stored in a memory device, such as thestorage device 24, or another storage device (not shown) within thevideo camera 10. Additionally, a central processing unit (CPU) (notshown) can be configured to execute the control routine. In someembodiments, the method 400 can represent a control routine implementedin a separate chip or chips (e.g., FPGA, ASIC, etc.). In someembodiments, the method 400 can represent a control routine stored in amemory device on a machine different from the video camera 10, such as avideo processing system or video editing system. The method 400 can beapplied to the processing of continuous video, e.g., frame rates ofgreater than 12, as well as frame rates of 20, 23.976, 24, 25, 29.97,30, 50, 59.94, 60, 120, 240 or other frame rates between these framerates or greater. For ease of description, the methods disclosed hereinwith reference to FIGS. 3-9 will be described as being performed by avideo compression system, such as the compression system 22 describedherein with reference to FIG. 1. However, one or more steps of adisclosed method may be performed by a particular component of the videocompression system, a different system or module, or a different device.Similarly, a single step of a disclosed method may be performed by acombination of components in the video compression system, a combinationof devices, or a combination of systems.

With reference to FIG. 4, the video compression system receives videodata comprising a plurality of video frames in block 405. The videocompression system can receive the video data from an image sensor, animage processing system, or a storage device on a video camera or otherdevice. For example, the video compression system can be implemented onboard a video camera and can be configured to compress video dataacquired with the video camera. In some implementations, the videocompression system is configured to compress video data received fromthe video camera in real time. The video compression system can receivethe video data from a video processing or video editing system. Forexample, after video data has been acquired and processed, a videoediting system can receive the video data and perform a compression onthe video data with the video compression system. This can be done, forexample, to convert the video data into a format suitable for editingwith the video editing system.

Progressing through individual video frames of the received video data,the video compression system divides the video frame into one or moreslices in block 410. As described herein with reference to FIG. 3, avideo frame can be divided into a plurality of slices, macroblocks,and/or blocks. Each slice can have an identical number of pixels,blocks, and/or macroblocks, and this can be the same for each slice ineach video frame or it can differ from slice to slice or from frame toframe. For example, a slice of a video frame can be made up of aplurality of blocks of image data, wherein each block includes 32×32pixels, 16×16 pixels, 8×8 pixels, or 4×4 pixels. In some embodiments,blocks can be arranged into macroblocks, wherein each macroblockincludes one or more blocks. The blocks in a macroblock can be arrangedsuch that the macroblock comprises a rectangle or square of pixels. Forexample, a macroblock can comprise 4 blocks each of 8×8 pixels, suchthat the macroblock can be a square of 16×16 pixels. The slice, then,can include one or more blocks or macroblocks and can have a size of,for example and without limitation, 128×16 pixels, 64×16 pixels, 32×16pixels, or 16×16 pixels.

In block 415, the video compression system groups slices of a videoframe into one or more sections. Each section can have a different size.Each section can extend horizontally and/or vertically across an entirevideo frame. The number of sections can be the same for each video frameor the number of sections can change. The sections can be used by thevideo compression system to make modifications to compression parametersduring compression of a particular video frame so that the encoded orcompressed size of the video frame is within a targeted range and/orless than a maximum video frame size.

Progressing through individual slices in the video frame, the videocompression system transforms a current slice in block 420. The videocompression system transforms the current slice using a DCT or DCT-liketransformation to transform information in the slice from the spatialdomain to the frequency domain. The result of the transformation can bea matrix or array of elements corresponding to magnitudes of frequencycontributions to the spatial information in the slice. Thetransformation operates in a manner similar to a discrete Fouriertransform. Transformation of a video slice is described herein ingreater detail with reference to FIG. 5.

In block 425, the video compression system calculates an entropy indexfor the transformed slice. The entropy index can correspond to the sumof the minimum number of bits needed to represent all the DCTcoefficients in a slice, representing the information content of theslice. An entropy index can be calculated separately for luma and chromatransformed values. The entropy index for a slice can be equal to acombination of the entropy index for each of the transformed luma andchroma values. For example, the slice entropy index can be equal to asum of the transformed luma entropy index and the transformed chromaentropy index. Calculation of the entropy index is described in greaterdetail herein with reference to FIG. 5.

In block 430, the video compression system generates a quantizationtable and quantizes the coefficients determined in the transformation ofthe slice in block 420. The quantization table can be based on astandard quantization table, wherein the standard quantization table ismodified for individual slices. In some embodiments, the quantizationtable can be the same for one or more slices. In some embodiments, thequantization table can be modified for each new slice. For example, thequantization table of the current slice can correspond to a standardquantization table multiplied by a function of the entropy indexcalculated in block 425. The quantized coefficients can be equal to theDCT coefficients calculated in block 420 scaled by a correspondingquantization table value. The quantized coefficients can be furthertruncated or rounded to be equal to an integer value. In someembodiments, the quantized coefficients are further encoded using acombination of Huffman coding, run-level coding, and/or arrangingcoefficients in a designated pattern (e.g., zig-zag scanning).

In block 435, the video compression system evaluates whether all theslices in a section have been compressed. If not, the video compressionsystem returns to block 420 to begin compression of the next slice ofthe video frame.

If the video compression system has compressed all the slices in asection, the video compression system evaluates whether all the sectionsin a video frame have been compressed in block 440. If not, the videocompression system modifies at least one compression parameter based atleast in part on an evaluation of the encoded size of the video frameversus an expected size of the encoded video frame in block 445. The atleast one modified compression parameter is then used to adjust thequantization tables for the slices in the next section. The videocompression system then returns to block 420 to compress the first slicein the next section. Modification of compression parameters is describedin greater detail herein with reference to FIGS. 6-9.

If all the sections have been compressed for a video frame, the videocompression system calculates a size of the compressed video frame andcompares that size to a targeted video frame size. Based on thiscomparison, the video compression system updates at least onecompression parameter in block 450 for use in compressing the next videoframe. The video compression system can then return to block 410 tocompress the next video frame of the received video data, represented bythe dashed arrow from block 450 to block 410.

The method 400 can be performed in a single pass for each video frame.The method 400 can be performed by the video compression system withoutthe use of an intermediate frame memory for storage of video frames thatare yet to be compressed and/or that have been previously compressed.The video compression system can be configured to perform the method 400in real time, such that a video frame is compressed at a rate that isgreater than or equal to the frame rate of the video data.

FIG. 5 illustrates a flow chart of an example method 500 of compressinga slice of a video frame. The method 500 can be used by the videocompression system as part of the method 400, described herein withreference to FIG. 4, or as a stand-alone video compression method.

In block 505, the video compression system transforms the slice of thevideo frame using a DCT or DCT-like transformation. The DCT utilized bythe video compression system can be similar to the DCT used in standardcompression schemes such as, for example and without limitation, JPEG,MPEG-1, MPEG-2, H.261, H.263, H.264/AVC, H.265/HEVC, Theora, MJPEG, etc.For example, a matrix of DCT coefficients, D, for a slice represented asa matrix of pixel values, M, can be calculated using the followingequations:D=TMT ^(T)  (1)where T is a matrix of the form:T _(ij)=1/sqrt(N) if i=0 andT _(ij)=sqrt(2/N)cos [(2j+1)iπ/2N] if i#0  (2)and N is the size of the slice (e.g., if M is an 8×8 matrix, N is 8). Insome embodiments, the matrix T can be configured to have integer valuesdivided by a fixed number that is a power of 2 (e.g., 1024, 2048, etc.).In some embodiments, the matrix of DCT coefficients, D, is calculated onblocks of 8×8 pixels, but blocks of different sizes can be used. Forexample and without limitation, blocks can be used with 4×4 pixels,16×16 pixels, etc. Other methods may also be used to determine thematrix of DCT coefficients.

The video compression system can transform the slice by transformingcomponents of the slice. For example, the slice can comprise a pluralityof pixel values for luma of the slice and a plurality of pixel valuesfor chroma of the slice (e.g., the luma and the chroma are components ofthe slice). The video compression system can determine DCT coefficientsfor the luma (Y′) of the slice and the chroma (Cb, Cr, Cb/Cr) of theslice. Similarly, the video compression system can determine DCTcoefficients for individual color channels (e.g., red pixel values ofthe slice, blue pixel values, green pixel values, etc.), pixel intensityvalues, pixel alpha channel values, etc.

In block 510, the video compression system calculates an entropy indexfor the slice of the video frame. The entropy index can be the number ofbits needed to represent the transform values for the slice. Forexample, if a transform value is equal to 5, the number of bits neededto represent that number is 3 (e.g., in binary 5 is equal to 101). Asanother example, the number 32 needs 6 bits (100000). The number 0requires 0 bits, as it can be discarded for compression purposes. Theentropy index can be calculated by aggregating the number of bits neededto represent each transform value in the slice.

In some embodiments, the entropy index is calculated separately for eachtransform within a slice. For example, where the video compressionsystem transforms the luma and the chroma separately (e.g., calculatingD_(l) and D_(c)), the video compression system can determine a lumaentropy index and a chroma entropy index. The entropy index of the slicecan be a combination of these entropy indices. For example, the entropyindex of the slice can be a linear combination (e.g., a simple sum) ofthe luma entropy index and the chroma entropy index.

In some embodiments, the entropy index is further modified by an entropymultiplier. The entropy index of the slice, the luma entropy index,and/or the chroma entropy index can be multiplied by an entropymultiplier. In certain implementations, each of the entropy index of theslice, the luma entropy index, and the chroma entropy index can have anindividual entropy multiplier used to scale the corresponding entropyindex. The video compression system can modify or adjust the entropymultiplier based at least in part on compression results from a previousslice, section, and/or video frame.

In block 515, the video compression system determines a quantizationmatrix, Q, for the slice. The quantization matrix can be based on astandard quantization matrix, Q. For example, the JPEG standardquantization matrix Q50 can be used as a basis for the quantizationmatrix, Q, used to quantize the transform coefficients, D, determined inblock 505. The quantization matrix, Q, can be determined for aparticular slice by scaling the standard quantization matrix, Q_(s), bya quantization scale value, Qscale. The value of Qscale can correspondto a product of the entropy index determined in block 510 and a factorcalled QscaleFactor. In some implementations, the quantization matrixcan be defined as:Q=Q _(s) *Qscale  (3)where Qscale is defined as:Qscale=(Entropy Index*QscaleFactor)/Target Slice Size  (4)The entropy index can be the total entropy index for the slice. Thetarget slice size can be a targeted size for the slice aftercompression. The value of QscaleFactor can be tuned or tailored toprovide desirable or targeted compression results. In some embodiments,the target slice size is represented as a number of bits used to expressthe compressed slice. In some embodiments, the quantization matrix, Q,can be based on a lookup table rather than using equations (3) and (4).

In block 520, the video compression system uses the quantization matrix,Q, to encode the slice. In some embodiments, the encoded slice is amatrix, C, with elements that are equal to the transform coefficients(e.g., the elements of the matrix D) divided by corresponding elementsfrom the quantization table (e.g., the elements of the matrix Q). Forexample, an element of the encoded matrix, C_(ij), can be defined as:C _(ij)=round(D _(ij) /Q _(ij))The slice can be encoded for each transform matrix determined in block505. For example, the luma and chroma transformations can each beseparately encoded. The rounding function can be used to truncatefloating point numbers to integers, in some implementations. Othersuitable mathematical functions may be used to reduce the amount ofinformation in the encoded matrix to achieve targeted compressionperformance.

In some embodiments, the slice is further encoded using a combination ofHuffman coding, run-level coding, exponential Golomb encoding, GolombRice encoding, and/or arranging coefficients in a designated pattern(e.g., zig-zag scanning).

FIG. 6 illustrates a flow chart of an example method 600 of adjustingvideo compression parameters after compressing a section of a videoframe. The method 600 can be used in conjunction with the method 400,described herein with reference to FIG. 4.

In block 605, the video compression system calculates a ratio of theactual compressed size of the video frame to this point to the expectedcompressed size. The actual compressed size can be a sum of the bitsused to represent the encoded slices (e.g., the matrix C described withreference to FIG. 5) within each previous section of the video frame.The expected compressed size can be defined as:Expected=Target Frame Size*Encoded Slices/Total Slices in Framewhere Encoded Slices refers to the number of slices encoded and TotalSlices in Frame refers to the total number of slices in the currentvideo frame.

In certain implementations, the calculated ratio for the frame andsection can be stored, such as in a storage device as described herein.The video compression system can use the stored calculated ratio from aprevious frame to determine whether to modify a compression parameterfor the current frame.

In block 610, the video compression system compares the ratio calculatedin block 605 to threshold values. For example, an upper threshold can be1.25 and a lower threshold can be 0.75. In this example, if the ratio ofthe actual compressed size to expected compressed size is between theupper and lower thresholds, no changes are made to compressionparameters. If the ratio is outside of the upper and/or lower thresholdvalues, a change can be made to compression parameters. In someembodiments, the threshold values can centered around 1 or at leastbracket 1, indicating that the compression is expected to achieve atargeted file size after encoding of the entire video frame. The lowerthreshold can be at least 0.5, 0.6, 0.7, 0.75, 0.8, 0.85, 0.9, 0.95, or0.99. The upper threshold can be less than or equal to 1.01, 1.05, 1.1,1.15, 1.2, 1.25, 1.3, 1.4, 1.5, or 2.

In block 615, the video compression system modifies the entropymultiplier described herein with reference to FIG. 5 if the calculatedratio of actual to expected compressed size is outside the thresholdrange defined by the upper and lower thresholds. Modifying the entropymultiplier has the effect of modifying the quantization table used toencode or compress the next section in the video frame. This can be usedto increase or decrease the amount of compression on the fly duringcompression of a video frame to increase the likelihood and/or to ensurethat the compressed frame size is within a targeted range.

The video compression system can use a lookup table to determine theentropy multiplier to use for the next video frame or next section in avideo frame. In some embodiments, the video compression system can beconfigured to compare a calculated ratio for the current frame to astored calculated ratio for the previous frame. If the calculated ratiosare similar, the video compression system can analyze the compressedsize of the previous video frame to determine whether to modify theentropy multiplier. For example, where contiguous frames are similar incontent, the result of the compression of the video frames should besimilar as well. Accordingly, if the calculated ratios for contiguousframes are similar at the same point in the compression process, it islikely they will have similar end results as well. Thus, the videocompression system can base a decision to modify a compressionparameter, such as the entropy multiplier, at least in part on a resultof the compression of the previous frame. If the previous frame was neara maximum frame size, the video compression system can decide toincrease the entropy multiplier (e.g., to increase the amount ofcompression).

In some embodiments, the threshold for the calculated ratio is set to 1.The video compression system can be configured to modify one or more ofthe compression parameters (e.g., entropy multiplier) where thecalculated ratio differs from the threshold by a predefined amount. Forexample, if the calculated ratio is within 5% of the threshold, thevideo compression system can be configured to leave the compressionparameters unmodified. If the calculated ratio is between 5% and 10%over or under the threshold, the video compression system can beconfigured to respectively increase or decrease the entropy multiplierby 5% (e.g., change the entropy multiplier from 1 to 1.05 if thecalculated ratio is 1.06). Similar ranges and changes can be programmedinto a lookup table. These ranges can be useful to reduce or eliminatechanges to compression parameters that may otherwise arise due to randomfluctuations in video frames.

In some embodiments, the video compression system can be configured tomodify the compression parameters based at least in part on the numberof slices that have been compressed. For example, early in thecompression process for a particular video frame, changes to thecompression parameters may be smaller than changes that occur later inthe compression process.

In some embodiments, the compression parameters can be modified at theend of a video frame based on the compression results from each section.For example, the entropy multiplier for a subsequent video frame can beset to a weighted average of the entropy multipliers for each section ofthe current video frame. The weight can be based at least in part on thenumber of slices in the section, the amount of information in thesection, the location of the section within the frame, or the like. Insome embodiments, the entropy multiplier is initialized to 1 at thebeginning of the compression process.

In block 620, the video compression system proceeds to the next sectionor video frame to continue the compression procedure. The next sectionor next frame can then be compressed using the modified or unmodifiedcompression parameters depending on the results of the comparison inblock 610.

FIG. 7 illustrates a flow chart of an example method 700 of adjustingvideo compression parameters after compressing a video frame. The method700 can be used in conjunction with the method 400 described herein withreference to FIG. 4.

In block 705, after compressing an entire video frame, the videocompression system calculates a total frame size for the compressedvideo frame. The total frame size can correspond to a total number ofbits used to represent the compressed video frame.

In block 710, the video compression system determines a size feedbackvalue based on a targeted frame size. The targeted frame size can be adesired, selected, or predetermined frame size represented as a numberof bits. The targeted frame size can be a range of sizes. The targetedframe size can differ from a maximum frame size that is used to set anupper limit on the size of a compressed video frame. For example, thevideo compression system can be configured to allow a video frame sizeto exceed the targeted frame size value but to not exceed the maximumframe size value. In certain implementations, the size feedback valuecan be a ratio of the target frame size to the total frame sizecalculated in block 710.

In block 715, the video compression system modifies one or morecompression parameters based on the size feedback value. For example,the video compression system can be configured to modify the entropymultiplier based on the size feedback value. The video compressionsystem can then use the modified entropy multiplier when compressing thesubsequent video frame. As described herein, the entropy multiplier canbe used to modify the entropy index of a slice, which in turn affectsthe quantization table and resulting compressed size of the slice.

By way of example, the video compression system can modify the entropymultiplier based on the following pseudo-code:

if (size feedback value <1)

-   -   Entropy Multiplier*=1+(1−size feedback value)*4.0;

else if (size feedback value >2)

-   -   Entropy Multiplier*=1+(1−size feedback value)*0.75;

else if (size feedback value >1.5)

-   -   Entropy Multiplier*=1+(1−size feedback value)*0.5;

else

-   -   Entropy Multiplier*=1+(1−size feedback value)*0.25;        Thus, the closer the size feedback value is to 1, the smaller        the change to the entropy multiplier. In some embodiments, the        video compression system can be configured to not change the        entropy multiplier unless the size feedback value is greater        than or equal to a threshold amount away from 1. This threshold        amount can be less than or equal to about 1%, 2%, 3%, 4%, 5%,        6%, 7%, 8%, 9%, 10%, 15%, or 20%. In certain implementations,        the entropy multiplier that is being modified is the weighted        average of entropy multipliers used to compress different        sections of the video frame.

In block 720, the video compression system proceeds to the next videoframe to continue the compression procedure. The next frame can then becompressed using the modified or unmodified compression parameters.

FIG. 8 illustrates a flow chart of an example method 800 of adjustingvideo compression parameters based on an accumulated size of acompressed video frame while compressing the video frame. The method 800can be used in conjunction with the method 400 described herein withreference to FIG. 4.

In block 805, during compression of a video frame, the video compressionsystem calculates the current compressed size of the video frame. Thecurrent compressed size of the video frame can be a sum of the bits usedto represent the size of all of the slices compressed in the currentvideo frame.

In block 810, the video compression system compares the currentcompressed size of the video frame to a maximum video frame size.Depending on the proximity of the current compressed size to the maximumframe size, the video compression system can modify one or morecompression parameters in block 815 so that the final size of thecompressed video frame is less than or equal to the maximum frame size.

For example, if the current compressed size of the video frame is X % ofthe maximum frame size, the video compression system can increase thevalue of Qscale (described herein with reference to FIG. 4), themultiplier for the quantization table, Q, by a predetermined, desired,or selected amount, y. By way of example, the video compression systemcan be configured to increase Qscale by 1 when the current compressedsize is 90% of the maximum frame size, to increase Qscale by 2 when thecurrent compressed size is 95% of the maximum frame size, to increaseQscale by 4 when the current compressed size is 97.5% of the maximumframe size, etc. Other methods of modifying Qscale can be used, such asusing different functions, lookup tables, or the like.

In block 820, the video compression system proceeds to the next slice tocontinue the compression procedure. The next slice can then becompressed using the modified or unmodified compression parameters.

FIG. 9 illustrates a flow chart of an example method 900 of adjustingvideo compression parameters based on an accumulated size of acompressed video frame relative to an accumulated size of a previousframe or frames after the same section within the frames. The method 900can be used in conjunction with the method 400 described herein withreference to FIG. 4.

In block 905, during compression of a video frame, the video compressionsystem calculates the current compressed size of the video frame. Thecurrent compressed size of the video frame can be a sum of the bits usedto represent the size of all of the slices compressed in the currentvideo frame.

In block 910, the video compression system compares the currentcompressed size of the video frame to the compressed size of theprevious video frame after an equivalent or approximately equal numberof sections, slices, macroblocks, or blocks had been compressed. In someimplementations, one or more previous frames can be used in thecomparison. For example, an average or median value of the compressedsize can be determined for a plurality of previous frames, thecompressed size being calculated after each slice and/or section in eachframe. Depending on the proximity of the current compressed size to thecompressed size of the previous frame (or frames) at the same (orsubstantially the same) point in the frame(s), the video compressionsystem can modify one or more compression parameters in block 915 (e.g.,by using the method 600 described herein with reference to FIG. 6). Thiscomparison can be done to avoid or reduce the likelihood of modifyingthe entropy multiplier too often during compression. This can lead to amore stable compression procedure with satisfactory results. In someembodiments, depending on the proximity of the current compressed sizeto the compressed size of the previous frame (or frames) at the same (orsubstantially the same) point in the frame(s), the video compressionsystem can use the compressed size of the previous frame or frames, orthe compressed size of the previous frame or frames at the same point inthe frame, to modify one or more compression parameters in block 915(e.g., by using the method 600 described herein with reference to FIG.6).

In some embodiments, two thresholds can be defined, T1 and T2. The firstthreshold, T1, can be defined as a ratio of the size of the previouslyencoded frame after the same section in the previous frame, Xp. If theencoded size of the current frame, X, is between Xp*(1+T1) andXp*(1−T1), then the compression parameters can remain unchanged. Thesecond threshold, T2, can be defined as a second ratio of the size ofthe previously encoded frame after the same section in the previousframe, Xp. If the encoded size of the current frame, X, is betweenXp*(1+T2) and Xp*(1−T2), then the compression parameters can be modifiedbased on the size of the previously encoded frame after the same sectionin the previous frame, Xp, rather than the expected image size, as inthe method 600. In some implementations, the first threshold, T1, can beless than or equal to 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08,0.09, 0.1, 0.15, 0.2, or 0.25. In some implementations, the secondthreshold, T2, can be less than or equal to 0.02, 0.03, 0.04, 0.05,0.06, 0.07, 0.08, 0.09, 0.1, 0.15, 0.2, 0.25, 0.3, 0.4, 0.5, or 0.75. Insome implementations, the second threshold, T2, is defined in terms ofthe first threshold, T1, or vice versa. For example, the secondthreshold can be set to T2=n*T1, where n is a number greater than 1.This multiplicative factor, n, can be, for example, at least 1.05, 1.1,1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, 2, 2.5, 3, 3.5, 4, 4.5, or 5.

Accordingly, the method 900 can be understood as a further refinement ofthe method 600. For example, rather than calculating the ratio of actualsize, X, to expected size in block 605 of the method 600, the videocompression system can calculate the ratio of the size of the previouslyencoded frame after the same section in the previous frame, Xp, toexpected size. This can then be used to determine whether to modify thecompression parameters in block 615.

After modifying the compression parameters, the video compression systemcan continue the compression procedure by proceeding to the next slice,section, or video frame in block 920.

Depending on the embodiment, certain acts, events, or functions of anyof the algorithms, methods, or processes described herein can beperformed in a different sequence, can be added, merged, or left outaltogether (e.g., not all described acts or events are necessary for thepractice of the algorithms). Moreover, in certain embodiments, acts orevents can be performed concurrently, e.g., through multi-threadedprocessing, interrupt processing, or multiple processors or processorcores or on other parallel architectures, rather than sequentially.

Conditional language used herein, such as, among others, “can,” “might,”“may,” “e.g.,” and the like, unless specifically stated otherwise, orotherwise understood within the context as used, is generally intendedto convey that certain embodiments include, while other embodiments donot include, certain features, elements and/or states. Thus, suchconditional language is not generally intended to imply that features,elements and/or states are in any way required for one or moreembodiments or that one or more embodiments necessarily include logicfor deciding, with or without author input or prompting, whether thesefeatures, elements and/or states are included or are to be performed inany particular embodiment. The terms “comprising,” “including,”“having,” and the like are synonymous and are used inclusively, in anopen-ended fashion, and do not exclude additional elements, features,acts, operations, and so forth. Also, the term “or” is used in itsinclusive sense (and not in its exclusive sense) so that when used, forexample, to connect a list of elements, the term “or” means one, some,or all of the elements in the list. In addition, the articles “a” and“an” are to be construed to mean “one or more” or “at least one” unlessspecified otherwise.

Conjunctive language such as the phrase “at least one of X, Y and Z,”unless specifically stated otherwise, is otherwise understood with thecontext as used in general to convey that an item, term, etc. may beeither X, Y or Z. Thus, such conjunctive language is not generallyintended to imply that certain embodiments require at least one of X, atleast one of Y and at least one of Z to each be present.

While the above detailed description has shown, described, and pointedout innovative features as applied to various embodiments, it will beunderstood that various omissions, substitutions, and changes in theform and details of the devices or algorithms illustrated can be madewithout departing from the spirit of the disclosure. Thus, nothing inthe foregoing description is intended to imply that any particularfeature, characteristic, step, module, or block is necessary orindispensable. As will be recognized, the processes described herein canbe embodied within a form that does not provide all of the features andbenefits set forth herein, as some features can be used or practicedseparately from others. The scope of protection is defined by theappended claims rather than by the foregoing description.

What is claimed is:
 1. An electronic device configured to compress aplurality of video image frames, the electronic device comprising: ahousing; an image sensor within the housing and configured to convertlight incident on the image sensor into video data, the video datahaving a frame rate and comprising a plurality of video frames includinga first video frame and a second video frame, the first video framecomprising a two-dimensional array of pixel values; and one or morehardware processors programmed to: compress a first section of the firstvideo frame using a discrete cosine transformation and a quantizationtable to thereby obtain a compressed first section, the first sectioncomprising a first plurality of slices, modify the quantization tablebased at least on a comparison of a size of the compressed first sectionand a first threshold to thereby obtain a modified quantization table,compress a second section of the first video frame using the discretecosine transformation and the modified quantization table to therebyobtain a compressed second section, the second section comprising asecond plurality of slices and being different from the first section,and store the compressed first section and the compressed second sectionto a memory device.
 2. The electronic device of claim 1, wherein the oneor more hardware processors is programmed to compress the first sectionby compressing each slice of the first plurality of slices using thediscrete cosine transformation and the quantization table to therebyobtain a plurality of compressed slices, the plurality of compressedslices together forming the compressed first section.
 3. The electronicdevice of claim 2, wherein each slice of the first plurality of slicesis divided into macroblocks, the macroblocks each comprising a different16×16 array of pixel values of the first video frame.
 4. The electronicdevice of claim 1, wherein the one or more hardware processors isprogrammed to: adjust the quantization table based at least on acomparison of (i) a size of the compressed first section and thecompressed second section and (ii) a second threshold, to thereby obtainan adjusted quantization table, compress a third section of the firstvideo frame using the discrete cosine transformation and the adjustedquantization table to thereby obtain a compressed third section, thethird section being different from the first section and the secondsection, and store the compressed third section to the memory device. 5.The electronic device of claim 1, wherein the first threshold comprisesan expected size of the first section with compression.
 6. Theelectronic device of claim 1, wherein the first threshold comprises aset percentage of a maximum size of the first video frame withcompression.
 7. The electronic device of claim 1, wherein the one ormore hardware processors is programmed to generate the quantizationtable by multiplication of a standard quantization table and an entropyindex.
 8. The electronic device of claim 1, wherein the first sectioncomprises a first set of pixel values that extend across an entirety ofthe first video frame, and the second section comprises a second set ofpixel values that extend across the entirety of the first video frame.9. The electronic device of claim 1, wherein the one or more hardwareprocessors is programmed to compress the first section a single timerather than a plurality of times.
 10. The electronic device of claim 1,wherein the two-dimensional array of pixel values comprises between 1920pixels and 4000 pixels in one dimension.
 11. The electronic device ofclaim 1, wherein the two-dimensional array of pixel values comprises atleast 4000 pixels in one dimension.
 12. The electronic device of claim1, wherein the one or more hardware processors is programmed to compressthe first video frame at a rate that is at least as fast as the framerate.
 13. The electronic device of claim 1, wherein the one or morehardware processors is programmed to decompress the compressed firstsection and the compressed second section for output and presentation ona display.
 14. The electronic device of claim 1, further comprising alens assembly supported by the housing, the one or more hardwareprocessors being positioned within the housing.
 15. A method ofcompressing a plurality of video image frames, the method comprising:converting light incident on an image sensor into video data, the videodata having a frame rate and comprising a plurality of video framesincluding a first video frame and a second video frame, the first videoframe comprising a two-dimensional array of pixel values; compressing,with one or more hardware processors, a first section of the first videoframe using a discrete cosine transformation and a quantization table tothereby obtain a compressed first section, the first section comprisinga first plurality of slices; modifying, with the one or more hardwareprocessors, the quantization table based at least on a comparison of asize of the compressed first section and a first threshold to therebyobtain a modified quantization table; compressing, with the one or morehardware processors, a second section of the first video frame using thediscrete cosine transformation and the modified quantization table tothereby obtain a compressed second section, the second sectioncomprising a second plurality of slices and being different from thefirst section; and storing the compressed first section and thecompressed second section to a memory device.
 16. The method of claim15, wherein said compressing the first section comprises compressingeach slice of the first plurality of slices using the discrete cosinetransformation and the quantization table to thereby obtain a pluralityof compressed slices, the plurality of compressed slices togetherforming the compressed first section.
 17. The method of claim 15,wherein the first threshold comprises (i) an expected size of the firstsection with compression or (ii) a set percentage of a maximum size ofthe first video frame with compression.
 18. The method of claim 15,further comprising: adjusting the quantization table based at least on acomparison of (i) a size of the compressed first section and thecompressed second section and (ii) a second threshold, to thereby obtainan adjusted quantization table; compressing a third section of the firstvideo frame using the discrete cosine transformation and the adjustedquantization table to thereby obtain a compressed third section, thethird section being different from the first section and the secondsection; and storing the compressed third section to the memory device.19. The method of claim 15, wherein the first section comprises a firstset of pixel values that extend across an entirety of the first videoframe, and the second section comprises a second set of pixel valuesthat extend across the entirety of the first video frame.
 20. The methodclaim 15, wherein said compressing the first section comprisescompressing the first section a single time rather than a plurality oftimes.